With the explosion of desktop publishing, the availability of digital media, and the advent of the World Wide Web (WWW), it is now possible to access large multimedia document repositories distributed throughout the world. Such access is becoming increasingly important in a number of applications including medical diagnostics, manufacturing, pharmaceutical research, surveillance, and distributed publishing of multimedia data from repositories. While data from widely distributed sources is becoming easily accessible, this also poses challenges in the design of multimedia databases as well as search agents at servers that now need to do intelligent site selection based on the integration of information from database sites and multimedia information in a query.
Accessing repositories in a distributed setting, either over a proprietary network or the more public Internet and its instantiation, the World Wide Web (WWW), poses quite a few challenges. In a typical scenario for such systems, the access to multimedia databases at remote web sites may be initiated by a client machine running a browser such as Netscape.TM. or Internet Explorer.TM.. The query is processed by the browser and sent to a web server. The web server selects the target multimedia database site and poses the query to the database in an acceptable form. The indexing mechanism of the database searches its repository for possible answers to the posed query. The answer is fed back to the web server for eventual relaying to the client. As can be seen from this scenario, system issues of security, consistency, versioning, persistence, etc., that are relevant for distributed access to traditional databases are also relevant for multimedia databases. More importantly, new technical issues arise due to the nature of the multimedia data. The traditional way of accessing such databases by assigning text annotations to image and video data are insufficient to unambiguously describe the media content, not to mention the time-consuming task of manually tagging the multimedia data. Automatic database creation and image content-based indexing, on the other hand, are difficult problems, for which effective methods are yet to emerge. While it is true that these problems exist also for multimedia databases designed for a stand alone use, they are felt more keenly when such databases are designed for use in a distributed (client-server) setting. Traditional methods of designing such systems using proprietary methods of query specification and data organization and search for specific fixed queries may not be suitable in such a setting where the remote user's queries may be unanticipated and referring to image content yet unextracted. Thus, there is a need for a coherent architectural framework for the design of such a database that admits incremental construction of the database in response to dynamically changing queries.
Even when individual databases are designed in a consistent manner, other issues of web-based access have to be taken into consideration. In particular, the process of selecting the relevant databases for a given query remains a challenging problem. Even though the number of multimedia databases linked by the network may be far fewer than text databases, it is still crucial to perform a careful selection of database sites for computational reasons, particularly because of the inherent complexity in image-content-based querying of each database. Secondly, if a query is posed to several databases, the answers may need consolidation and summarization before they can be presented to a user. Finally, the transformation of a user query (who may not be aware of the databases and their capabilities) into a form suitable for querying the remote multimedia databases may require eliciting more information from the user than that provided in the original query.
Although commercial systems are being developed that allow multiple text database to be accessed over the web via SQL, ODBC or Perl gateways, methods do not seem to currently exist that allow a user to interact with multiple remotely located multimedia databases. The system that come nearest is the University of Chicago's Webseer.TM. system. It is modeled after traditional search engines such as Lycos.TM. and Alta Vista.TM. in that it also creates its own indexed database at the web server site by navigating known web sites and recording text as well as image related information. More common are systems, such as Webseek.TM., Berkeley Digital Library.TM., that allow web users to use their pre-designed image databases by connecting to their specific URL. Such systems often reflect closed design strategies that are optimized for handling fixed a set of queries using representations all precomputed at the time of database creation. Being client-centric, they offer little capability for sharing and a potential for duplication and inconsistency.
Previous work does exist, however, on addressing individual aspects of web-based multimedia databases, particularly, in the design of image and video databases, and resource discovery systems. Issues of image data modeling, image matching and classification, similarity metrics for image content matching, the design of efficient data structure for image content representation, and visual query languages have all been explored. These issues were, however, examined in isolation in the context of the related applications with no coherent design framework emerging. Also, because most of the work in this area has come from the computer vision community, the image database itself has been treated as a list of image files or their abstracted representations. Commercial implementations have worked on integrating them into traditional databases such as relational as BLOBS (binary large objects), object-oriented databases using text annotation, and more recently, supporting content-based querying in object-relational systems such as in Illustra as described in an article by M. Ubell and M. Olson entitled "Embedding image query operations in an object-relational database management system," Proceedings SPIE Conference on Storage and Retrieval of Image and Video Databases, pp. 197-203, 1995. Even so, only a few approaches have attempted to establish a theoretical framework for multimedia database systems, notable among these is the work on structured multimedia database systems based on a mathematical model of a media instance as discussed by S. Marcus and V. S. Subrahmanian, "Foundations of multimedia database systems", Journal of the ACM, 43(3), pps. 474-523, 1996. In these systems, the integration of multimedia data in traditional database frameworks has focused mostly on providing image-content querying capabilities by suitable encapsulation of image-based search in the respective objects. An important aspect of such multimedia databases is also the variety of representations and image data organization schemes they employ in handling queries. The suitability of traditional databases in supporting such representations is yet to be investigated.
In order to make multimedia databases web-based, a critical issue is how the capability of such databases (metadata) can be described to enable a web server to perform an intelligent selection of relevant databases in response to user queries. Most of the site selection work has been focused towards handling text information. For example, web search engines such as Lycos.TM. and Alta Vista.TM. currently create web indices in their search engines by periodically scanning potential web sites and using the text information in their resident HTML pages. But most implementations of text-based distributed systems do not perform site selection, often posing a query to all sites in parallel as done in the CLASS (College Library Access and Storage System) and the NCSTRL (Networked Computer Science Technical Report Library) systems at Cornell University as discussed by C. Lagoze and J. Davis "Dienst: An architecture for distributed document libraries" Communications of ACM, 38(4), pg. 47, April 1995. More recently, techniques from information retrieval are being used for intelligent resource site selection. Examples of such systems include GLOSS (from Stanford) as discussed by L. Gavarno and H. Garcia-Molina, "Generalizing Gloss to Vector-Space Databases and Broker Hierarchies," Proceedings of the 21st International Conference on Very Large Data Bases, pages 78-89, 1995, WHOIS++ (from Bunyip Corporation) and HARVEST (from University of Colorado) as discussed by Michael Schwartz, "Internet Resource Discovery at the University of Colorado", IEEE Computer, pages 25-35, 1993. These systems employ statistical approaches to record the frequency of occurrence of text keywords from known sites to construct an index of relevant sites for directing a query. In particular, the GLOSS (generalized glossary of servers) server keeps statistics on the available databases to estimate which databases are potentially most useful for a given query using Boolean and vector-space retrieval models of document retrieval. Another approach has been to use inference networks for the text database discovery problem. This summarizes databases using document frequency information for each term together with the inverse collection frequency of different terms. An inference network then uses this information to rank the databases for a given query. Finally, the HARVEST system provides a flexible architecture for accessing information over the Internet using "Gatherer" modules to collect information about the data resources which is passed to the "Broker" modules. A structured representation of these broker modules is kept in the Harvest server registry which, in a sense, becomes the meta-database exposing information about the individual database sites.
The present invention employs network capabilities to achieve various advantageous ends. The previous references are intended to provide a background for any appropriate network implementation required by the disclosed embodiment below for the purpose of rendering or transporting database information. Disclosures of all of the references cited and/or discussed above in this Background are incorporated herein by reference.